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ABSTRACT 

When students choose postsecondary vocational 
education programs, they pay, in both money and in income lost during 
the time they are in the program. The public also pays by tax support 
for many educational programs. However, students often have little 
information about the effectiveness of these programs in terms of job 
placement, wage rates, and career potential. Some efforts to 
determine and disclose these outcomes have been made by Florida r^a 
Arizona, These states used Social Security numbers and state 
unemployment files to follow up on vocational education program 
completers* Although this method has some deficiencies, it is fairly 
simple and not too costly. Arizona attempted to do such a study and 
publish the results. However, missing information such as type of job 
obtained, net jUst employment oi::tained, undermined the results. 
Finally, political pressures contributed to the demise of the 
information disclosure program. It is recommended that future 
: .-if ormation disclosure efforts take into account the effect of the 
disclosures on school systems and students and the means of 
distribution before such a program is undertaken. (KC) 
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INTRODUCTION 



Students are consumers of vocational education, When they choose a vocational program, 
they purchase skills, knowledge, a credential, and access to future employment opportunities. 
The purchase is made with two currencies: first, time away from work and other training 
programs and second, money. The public shares in the dollar cost through support of public 
schools and fmancial aid programs. This paper is prompted by a concern that students often 
make their choices with limited information about the quality of the vocational programs they 
enter and the employment prospects in the occupations they choose. It discusses how improving 
information about the expected outcomes of educational alternatives might improve the quality of 
the individual student's choice and the quality of postsecondary vocational education as a whole. 

The basic argument is quite simple. Prospective students can choose whether or not to 
attend a postsecondary institution at all and, if they decide to do so, can choose the institution and 
the progranL With complete understanding of the benefits that different institutions and programs 
have to offer — such as the likelihood of completing the program, the employment opportunities if 
they complete the program, and salaries they are likely to earn if they get a job related to their 
training — students will choose institutions with good expected outcomes. Institutions have an 
incentive to improve these outcomes if funding follows student enrollment, either through 
student tuition (often federally subsidized through loans and grants) or through funding based on 
enrollments. Efficient allocation of resources is therefore fostered by maximizing the amount of 
infonnation on outcomes available to students. 

Obviously, it is not possible to provide complete information, and even if it were, students' 
choices are constrained by many factors that would prevent them from always choosing the best 
program. Nevertheless, we believe that if more attention weve paid to disclosure of relevant 
information, we could have a more efficient allocation of educational resources than now exists. 
Student choice, if properly informed, could be a powerful mechanism to induce institutions to be 
responsive to changing occupational opportunities and to demands for quality education. 

In this paper we first review past use of information disclosure to consumers generally and 
to consumers of vocational education specifically. Next, wc discuss the potential benefits and 
costs of greater information disclosure in vocational education and describe the conditions that 
must be met for information disclosure to promote -nore a efficient allocation of resources. Wc 
then examine Arizona's experience with information disclosure. Finally, we draw some 
conclusions about the usefulness of an informatior. disclosure program such as Arizona's. 

1 
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INFORMATION DISCLOSURE FOR CONSUMERS 



Economists and policy makers have long recognized the potential benefits of information 
disclosure. The efficiency of any market depends on consumers having full information about the 
quality and expected performance of what they buy. Only when consumers know what they will 
get for their money can they encourage the production of the best products through their 
patronaf- Yet, in many markets, consumers are poorly informed about the quality of products 
before buy. To help consumers improve their choices, the government has passed many 
different information disclosure laws over the years. 

Some information disclosure laws protect consumers from the dangers of being ill- 
informed. For example, there are laws that requirr certain non-prescription drugs to carry labels 
warning people with certain medical conditions not to use the medicine. Similarly, there arc laws 
that require plastic bags to cairy warnings against using them in infants' beds, and laws that 
require labeling of flammable materials to make sure consumers know that the materials can 
easily catch fire. 

Other information disclosure laws help individual consumers make the choices that best 
meet their personal needs. For example, labels on food products identify ingredients and 
nutritional content to help consumers select foods suitable for their diets. Tnith-in-lcnding laws 
require financial institutions to disclose effective interest rates, making it easier for borrowen to 
compare the costs of different loans. Sometimes consumer information is disclosed voluntarily. 
Responsible commereial advertising, for example, tells consumers something about the nature of 
products, and consumers can use this information to decide which purchases will fill their 
individual needs. 

Studies show that some information disclosure laws have more impact than others on 
consumer behavior. In reviewing research on the impact of consumer information disclosure 
laws, Friedman and Sugarman discovered that some information disclosure laws have had little 
effect 1 For example, the tnith-in-lcnding laws requiring disclosure of effective interest rates by 
lending institutions appear to have had little impact on consumer borrowing habits, although 
consumers feel more confident in their decisions because of the information. The financial 
prospectus required by the 1934 Security and Exchange Act is rarely read because it docs not 
contain the information consumers consider most important to their investment decisions. 



'Lee S. Friedman and Stephen D. Sugannan, School Sorting and Disclosure: Disclosure to Families as a School 
Reform S.rategy, Part One, Beficeley, Graduate School of Public Policy, Working Paper #139, p. 56-62. 



Other disclosure laws have had more impact on consumer behavior. Research shows that 
consumers are aware of unit pricing requirements and do use the information to guide their 
purchases. Friedman and Sugarman conclude that carelessly designed and implemented 
disclosure schemes can easily fall short of their goals. The important lesson is that the success of 
disclosure depends at least partly on the relevance, simplicity, and timing of information 
disclosed. 

INFORMATION DISCLOSURE IN EDUCATION 

Disclosure of information about education programs became a policy issue following the 
enactment of major financial aid programs. The 01 Bill (Veterans Readjustment Act of 1944) 
placed, for the first time, significant fmancial power in the hands of millions of potential 
students. Abuses followed because the program created incentives for institutions to enroll as 
many students as possible, regardless of their chances for success. Investigations found cases of 
educational enteiprises compiling phony figures on veterans* enrollments, engaging in fraudulent 
advertising and sales practices to attract students, and going out of business and moving across 
state lines to reopen.2 These abuses led Congress to develop safeguards to protect educational 
allowances. 

Federal financial aid programs introduced in the 1960s and 1970$ — the largest of which 
was the Guaranteed Student Loan (GSL) program — also put large amounts of money in the 
hands of students who could fall victim to harmful recruiting and enrollment practices. High loan 
default rates among students who failed to complete programs led the federal government to 
restrict institutions* eligibility for loan programs and revise program regulations. The new 
regulations required institutions to maintain records on the admission, attendance, progress, 
placement, and indebtedness of loan recipients. They required institutions to establish a fair and 
equitable refund policy for tuition, room, and board and to distribute this policy to all students. 
They also required institutions to make available to all prospective students, before obligating 
them to pay any tuition or fees, accurate ir ormation concerning current academic or training 
programs, faculty, and facilities. The regulations for vocational programs specified that this 
information must include the percentage of recent graduates who obtained positions in the areas 
for which they were prepared and their average starting salaries. Finally, to keep institutions 
from admitting students indiscriminately, the regulations required institutions to show a 



2joan S. Stark, "The Emerging Consumer Movement in Education," In Promoting Consumer Proitction for 
Students: New Directions for Higher Education, San Francisco, Jossey Ba.ss, 1976, p. 1-8. 



substantial and rwsonablc basis for concluding that each prospective student had the ability to 
benefit from the instruction or training offered^ 

The Department of Education remains concerned about these education consumer issues, 
particularly in vocational-technical programs. The Department recently commissioned a study to 
investigate a number of issues related to high GSL default rates at vocational-technical schools: 
admission of unquaUfied students who are not adequately informed about the difficulty of the 
material they must master, students who drop out of the program or who do not get jobs they 
were trained for, misrepresentation of program completion and job placement rates during the 
recruiting process, and lack of fiscal integrity. The study found what Secretary Bennett termed 
"excessive evidence" that private, for-profit trade s hools are exploiting and deceiving students.* 

POTENTUL BENEFITS AND COSTS OF MORE INFORMATION 
DISCLOSURE IN VOCATIONAL EDUCATION 

Disclosure of information on outcomes could protect students from the kind of abuses 
discovered in the GI educational allowance and GSL programs. If students were provided with 
outcome information such as completion rates, job placement rates, and entering wage rates for 
different programs and institutions, they could avoid programs that offer them little for their 
investment of time and money. As indicated above, some private schools have been charged with 
false and misleading advertisement about the success of graduates. Independently collected 
information on outcomes could be used to disprove false advertisements. 

In addition to protecting students from unscrupulous behavior on the part of institutions, 
more information on outcomes could help students make the best choices for themselves from 
available options. At the present time, students are forced to choosft schools with limited 
objective information about the quality of programs and expected outcomes. Program evaluation 
results and other information collected by the state or individual institutions arc rarely available to 
the public. 

Students typically rely on high school counselors, friends, family, and program 
advertisements to help them choose among educational options and between education and work. 
Like consumers purchasing food for the family or choosing a loan, students could benefit from 
accurate infonnation about their options. Objective information on outcomes would give students 

^Robert H. Davidson and Joan S. Stark, "The Federal Role," in Promoting Consumer Protection for Students: 
New Directions for Higher Education, San Francisco, Jossey Bass, 1976, p. 10. 

* Brian Fitzgerald and Lisa Harmon, Consumer Rights and Accountability in PostserorJary Vocational-Technical 
Education: An Exploratoty Study. Washington, D.C. **elavin Associates, February 1988. 
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a more realistic picture of the likely bcncfitt of attending different schools and entering different 
occ upations. If students were able to assess whether the costs of an education were justified by 
futire employment and earnings prospects, perhaps many would avoid incurring large debts they 
were unable to pay because they could not fmd adequate employment after completing school. 
More formal information disclosure policies would especially benefit disadvantaged students 
who are less likely to have access to accurate informal sources of information on program 
quality. 

More information disclosure could improve the quality of vocational offerings as well as 
help students make wise personal decisions. Student choice plays an important role in 
determining what programs are offered and by which institutions. Wticn students select the best 
programs, an enrollment-based funding system will direct vocational education funds to high 
quality programs and away from low quality ones. It will also direct additional public funds in 
the form of student financial aid. 

Institutions as well as students could use information on outcomes. By comparing their 
outcomes with those of other institutions, they could identify immediately which programs 
needed improvement and not wait for students to "vote with their feet** The information on 
outcomes currently available to institutions is limited. Most institutions conduct their own 
evaluations and participate in peer reviews for accreditation purposes, but although this 
information provides feedback for planning and program improvement, it is expensive to collect, 
does not readily allow comparisons among institutions, and is not available on a regular basis. 

Information disclosure does not come without costs, however. Against the potential 
benefits described above must be weighed the costs of collecting and disseminating information, 
which could be substantial. A statewide effort requires collection of information Lbout thousands 
of graduates from possibly hundreds of vocational programs. Following the collection, the data 
must be interpreted and di iseminated to those who can benefit from the information, including 
institutions, education agencies, and potential students. 

Critics of information disclosure are quick to point out another potential cost of publishing 
completion, placement, and wage data. There is a danger that institutions att/;mpting to perform 
well on such measures might increase admissions requirements so a higher proportion of 
students who enter could complete and find jobs. This kind of response to an infonnation 
disclosure program could reduce services to disadvantaged and high risk groups. 
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NECESSARY CONDITIONS FOR EFFECTIVE INFORMATION DISCLOSURE 



Information disclosure laws hold the promise of improving student choice and forcing 
institutions to become more performance -oriented For this promise to be fulfilled, a number of 
conditions have to be met 

• institutions must generate relevant, reliable measures of performance tha<. allow 
comparisons among programs and institutions; 

• information on performance must be disseminated to potential students: 

• students must take the information into account when making their choices; 

• institutions must change their programs in response to the information. 
The prospects for meeting these conditions are discussed in this section. 

Performance Measures 

Students can make wise choices of schools and programs based on performance measures 
only if those measure are relevant, easy to understand, reliable, and comparable among 
institutions. Program completion rates, placement rates, and expected wages appear to be the 
most available measures of vocational outcomes. While this information would easily meet the 
criteria of relevance, understandability, and comparability, there are both conceptual and practical 
problems in providing it to prospective students. 

Placement Rates 

Placement rates — defined as the percentage of completers who find jobs in occupations 
related to their training — have long been considered a relevant standard for evaluating the 
performance of vocational education. Many vocational educators, however, have several reasons 
for objecting to an emphasis on placement rates. First, they maintain that the emphasis on 
placement fails to distinguish vocational education from job training. Adopting placement as the 
primary criterion ignores the multiple goals of vocational education. In addition to imparting 
specific job skills, vocational education is also concerned with the acquisition of basic skills in 
reading, writing, and mathematics. It also seeks to deliver general vocational skills that will serve 
students in a variety of ways as their careers advance. 



Second, vocational educators argue that the employment of vocational education students is 
detemuned by a large number of economic and personal factors beyond the control of the 
vocational education sy stent "Hold us accountable for emphyability, but not for employment* 
has been a frequent refrain.^ 

Third, many vocational educaton express concern that a single-minded focus on placement 
encourages programs to admit only those students who are easiest to train and place. Important 
objectives — such as serving students with special needs and opening certain occupations to 
women, minorities, and the handicapped — arc ignored by pure placement standards. In practice, 
most postsecondary vocational institutions have enough places for all interested students able to 
meet the minimum qualifications. Therefore, as a practical matter, most programs arc not very 
selective. But certain programs— especially those with the greatest potential for job placement 
and highest potential wages — regularly have more applicants than they can accept, and there is a 
danger that these programs would ignore hard-to-serve students if the programs were held 
accountable strictly for placement 

In addition to these conceptual problems, good data on placement are expensive to obtain 
and often unreliable. The Vocational Education Data System mandated in the 1976 Amendments 
to the Vocational Education Act required states to follow-up students six months after they 
completed or left a program. These surveys have had notoriously low response rates and 
therefore have provided very limited and biased information about the success of programs. 

Completion Rates 

Focusing on completion rates rather than placement rates solves some of the problems 
encountered with placement rates. The completion rate — the percentage of students who have 
earned a degree or certificate in a recognized program — is a measure of employability, and 
vocational educators are more willing to be held accountable for employability than employment. 
In addition, completion data are much easier to obtain than placement data. 

It is appropriate to reward institutions for high completion rates because students who 
complete programs are more likely to get good jobs in fields related to their training than are 
students who do not complete. It is important to keep in mind, however, a high completion rate 
does not necessarily mean that the program is good or that it should be offered at all. If the 
quality of instruction in a program is poor, students may not be employable even if they complete 

^ Gerry Hendriciuon, Evaluating Vocational Education: The Federal Stimulus. Vocational Education Study 
Publication No. 5, Washington. DC. National Institute of Education. March 1981, p.7. 
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the program. Furthermore, because employment opportunities depend on national and local 
economic conditions, the best training in the world will not guarantee en^ployment after 
completion if there arc no job openings. It can legitimately be argued that if the demand for labor 
in a particular field is weak institutions should not offer training in that area, no matter how 
exemplary the training or how high the completion rate. In sum, holding institutions accountable 
for completion does no*, force them to offer high quality programs or to be responsive to demand 
for labor in various occupations, whereas holding them accountable for placement does. 

Wages 

Information on expectid wages for program completers would be extremely valuable to 
potential vocational students, particularly if it included data on long term income prospects for 
workers in particular occupations as well. The problems with expected wages as aii outcome 
measure are practical rather than conceptual. Data on wages obviously must be collected after 
students have completed their programs and been employed. As indicated in the discussion on 
placement data, institutions have had little success in collecting information from students after 
they have graduated 

Unemployment Insurance Files as Information Sources 

State- maintained unemployment insurance files are potentially useful sources of 
information on student outcomes. These files can be used to determine whether or not specific 
individuals have been employed during a particular quarter and, if they have, the number of 
weeks worked and their earnings for the quarter. Because employers must pay unemployment 
insurance taxes for all their employees and because information on individual-level employment 
and earnings are used to calculate benefits, unemployment insurance files are a highly reliable 
source of placement information. They can also be used to determine employees' quarterly 
earnings. At least three states — Arizona, Florida, and Kansas — have used their unemployment 
insurance fiies to track former vocational students. 

Although reliable, unemployment insurance files have some limitations. Only completers 
who become employed in jobs covered by unemployment insurance can be tracked. Completers 
who are self-employed, join the military, take out-of-state jobs, or enroll in another educational 
institution are not included in unemployment insurance data bases. Placement rates calculated 
from these data bases alone will therefore understate the true placement rates, especially if 
postsecondaiy enrollment and military service are considered placements. Florida has overcome 
this problem to a large extent by searching for the social security numbers of former vocational 
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students in data bases maintained by the military, public four-year universities, and community 
colleges.^ Although searching in more than one data base obviously adds to the cost of tracking 
former students, it can be done as long as social socurii^ numbers are used for identification. 

A second problem with u:?empIoyment insurance files is that they do not contain 
information on employees' occupations. As a result, it is often difficult u) tell if an individual is 
in a job related to his .or her training. This limitation can al&o be overcome with some additional 
effort Florida surveyed the employers of the former students found in the unemployment 
insurance files and asked them to idenH'y the employees' occupations. The response was 
excellent— occupational infonnation was obtained for 75 percent of the former students. 

A thiid limitation of unemployment insurance files relates to the wage infonnation. The true 
entry -level wage rates for different occupations cannot be obtained from unemployment 
insurance data. In most states, employer reports indicate the number of weeks in which the 
employee worked and the total wages paid, but not the acmal hours woiked. Because they do not 
indicate the total number of hours worked, the files cannot be used to determine wage rates. 

As a source of information on placement and earnings, unemployment insurance files have 
a clear cost advantage over student follow-up surveys. Searching for the social security numbers 
of completers in computerized unemployment insurance records is much easier and less costly 
than conducting a survey of completers. All that institutions have to do is co'^ect social security 
numbers from students and indicate the occupational codes of their programs. This approach also 
yields much more information than surveys usually do — response rates for follow-up surveys 
tend to be very low. Arizona has already used its unemployment insurance files to publish 
placement and earnings data for program completers in all public and private vocational 
programs. Kansas has compiled similar data, but only for occupational areas, not for individual 
institutions. Florida has gone the furthest in using other state-level data as well as unemployment 
insurance data to track students and has combined this information with information from 
surveys of employers. In Florida, the data is used by the Department of Education for planning 
and evaluation of vocational education and — at least yet — been not used to provide information to 
the potential students. 

Using unemployment insurance data for the purposes described here docs raise some 
privacy issues. There is a growing public concern about what information the government and 
other organizations maintain and how they use it Although only aggregated information would 

^Thesc efforts are described in Occupational Jdemifier Project: Legislative Report, Department of Education, Sute 
of Florida. April 29, 1988. 
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be published, the idea of usfjig social security numbers to track individuals' employment may be 
objectionable to some. Sensitivity to this issue and assurances that dau will not be published 
when there axe only a small number of graduates should help alleviate such concerns. 



Dissemination 

Institutions with jsood records would want to make their successes known and could be 
relied upon to advertise their job placement and wage rates, but institutions that have not 
performed well would have no incentive to cooperate in disclosing information. Therefore, 
disseminatjcr" of information on educational outcomes should be a state-level responsibility. 

Among those who should receive the information are the state legislature, the governor's 
office, state agencies with responsibility for education and economic development, 
postsecondaiy vocational institutions, high school counselors, high school students, and 
members of the public who request it High school counselors can advise potential vocational 
students who are still in school, but other ways must be found to reach those who are not. 
Publication of results in newspapers and distribution of brochures in public places such as 
libraries are possible ways to reach wide audiences. 

The form in which the information is presented is important As Friedman and Sugarman 
pointed out, consumeis are much more likely to respond to information that is simple as well as 
readily accessible.'^ Attractive brochures that are brief and to the point are important 

To be useful, the infonnation must also be disseminated in a timely manner. Because there 
has to be a lag between program completion and collection of outcome data (to give comp'etcrs 
time to find jobs and establish a salary history), data will necessarily be at least a year or a year- 
and-a-half old by the time it is published. Every effort must be made to ensur: that the lag is not 
any longer than necessary because the value of the data declines rapidly over time. Three or four- 
year-old wage and job information would be of little interest to potential students. 

Student Choice 

Assuming students had access to comparative information on outcomes for different 
programs and institutions, would they use it to make their enrollment decisions? For example, 
would an individual considering a career in electronics study use the information to determine 
which electronics programs had the highest placement rates and wages and then enroll in one of 

^Friedman and Sugarman, Op. Cii., p. 56. 
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them? This question is difficult to ^swer» because attendance decisions are complex. 
Nevertheless, it is an important question, because the potential benefits of information disclosure 
cannot be realized if students do not actually use the information. 

There is some evidence that high school senion do in fact care about job placement records 
in choosing the colleges they attend, although it is not specific to vocational students. A 1982 
survey of a national san^le of graduating seniors asked about the importance of various factrrs 
in choosing the college they planned to attend, and one of the factors was the job plac^^ment 
record." A total of 41 percent reported that the job placement record was somewhat important, 
and 48 percent said it was very inq;x>rtant Only 12 percent said it was not inq)ortant The sample 
included all students attending or planning to attend college, not just vocational students. It is 
reasonable to predict that vocational students would be even more interested in job placenient 

Students als3 care about colleges' reputaticms in academic areas. The same survey found 
that 49 percent of the students rated academic reputation as very important in their decision ?nd 
44 percent ranked it as somewhat iraportant Interestingly, in a separate study, Manski found that 
students do not predictably choose the highest-quality school they can get into. They arc most 
likely to choose schools with SAT averages about 100 points above their own.^ 

Despite apparent student interest in outcomes and overall quality, there are a number of 
factors that limit student choice. We cannot assume that students are automatically free to choose 
the best programs and schools. Location is an obvious limiting factor. Many vocational students 
have to live at home while enrolled, often for fmancial reasons. Even if they are amed with 
infomation that the local program is not one of the best, they may still have to choose it Another 
limiting factor is student ability — if the best programs are full, not all students can choose them; 
some will have to choose inferior ones. Cultural factors may make it very difficult for students to 
select certain programs. Some programs are traditionally dominated by males or females and may 
not be chosen by the other sex even in die face of evidence that they prepare students for jobs 
with high wages. More than information alone is needed to steer these students in the direction of 
high quality programs. Still other limiting factors for stui ents are the cost of the program and the 
availability of fmancial aid. Some students may have to choose the institution with the lowest 
costs or with the best financial aid offer. 



ERJC 



^High School and Beyond 1980 Sophomore Cohort First Follow-Up (1982). Data File Users' Manual. National 
Center for Education Statistics, p. 147. 

'Charlss F. Manski and David A. Wise, College Choice in America. Cambridge, Harvard Univenity Press, 1983. 
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In sum» it would be rational for students to choose the institution with the best expected 
outcomes, and there is some evidence that students take into account expected outcomes in 
making their choices. However, many factors enter into student choice, and we have no real way 
to predict how much weight students would give tc published information on outcomes. 

Institutional Response 

If informatioa disclosure is to affect the quality of vocational education, institutions musi 
improve their programs because of (a) the comparative information showing how they are doing 
relative to other institutions and (b) student demand, which presumably has been influenced by 
the information. In addition to improving <he quality of the programs they have, tiiey must 
eliminate programs for which demand is low, expand programs that are doing 'veil, and possibly 
add new programs in fields that are doing well in other institutions. 

Organizations tend to resist change, ai\d it is easy to imagine that at least some institutions 
would ignore the information. Postsecond;iry vocational institutions can be unresponsive to 
changing technologies and changing demand for workers in different occupational areas. 
Opening and closing programs requires costly investments in new equipment and personnel, and 
existing faculty cannot easily be reassigned to teach different skills. Consequently, institutions 
often continue to offer outdated programs as long as students are willing to enroll. With more 
information disclosure on outcomes, institutions would have to be more responsive or risk losing 
their funding. 

To this point, we have focused on the potei:*ial benefits of an information disclosure policy 
and examined the conditions that have to be met for an effective information disclosure system. 
In the rest of the paper, we describe Arizona's experience with information disclosure to 
highlight the practical considerations involved 

Mi IZONA'S EXPERIENCE WITH INFORMATION DISCLOSURE 

In 1985, Arizona imple.nented an innovative information disclosure program. The state 
collected from each postsecondary vocational institution the social security numbers and 
programs of all students completing programs during the 1982-83 school year. The social 
security numbers were checked against unemployment insurance data files to determine whether 
or not the completers were employed during the year beginning in the fall of 1983 and if so, what 
their wages had been during the year following their completion. The results were published and 
distributed to high school counselors and to vocational institutions. 
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Unfortunately, the information disclosure program only lasted one year. The law 
a ithorizing the data collector, and publication remains on the books, but the legislature did not 
authorize any funding after the first year. A mix of technical and political problems were 
responsible for the program's demise, but valuable lessons were learned during the one year of 
operation. Many in the state considered the project a success and believe that it generated valuable 
information for students, educational institudons, and the state. 

To gather information for this paper, we interviewed the major participants in the design 
and operation of the prosram, including a former aide to Governor Babbitt, the director of the 
Vocaional Education Resources Coordinating Unit at Northern Arizona University (NAU), 
which wijs responsible for collecting the social security numbers and program identification 
codes and publishing the report, the director of Research and Administration in the Arizona 
Department of Economic Security, which maintains the state unemployment insurance data base, 
and the economic consultant oh the project Through these interview? we were able to find out 
how an information disclosure system worked and to determine some of the practical problems 
likely to be encountered in a program of this type. 

Rationale for the Program 

The idea of a report for students, parents, and schools originated in Governor Babbitt's 
office. The legislature was considering a larger vocational education initiative as part of a package 
of economic development legislation. As a condition of support for the package, the Governor 
insisted on articulation and accountability measures for the state's education systems. For 
postsecondary vocational education, tlie Governor specifically wanicd performance measures. 

By providing objective information about the percentage of students completing both public 
and private programs, the Governor hoped that students would make better choices among 
vocational options and have realistic expectations about the outcomes of completing different 
programs and attending liffercnt institutions. The Governor also hoped that the information 
would protect students. State leaders were very concerned that the placement and wage rates 
advertised by some private training institutes were suspiciously high, and they wanted an 
independent source of information that would discourage false advertising. 

Initial Objections to the Information Disclosure Program 

When the proposal to disclose outcome information based on unemployment data was 
introduced, opponents quickly appeared. The piimaiy opponents were the conmiunity colleges, 
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which were concerned that the data would be insensitive to justifiable differences among 
programs, would be so inaccurate that potential students would be misled, would encourage 
creaming as an enrollment practice, and would have limited value to students and the state. 

Differences Among Programs 

Many people thought that comparing completion and placement rates of programs in the 
same occupation at different schools was inherently unfair. They argued that different schools 
have different resources, and that small schools with limited resources cannot provide the same 
diverse curriculum as larger schools. They argued further that publicizing placement data would 
highlight regional differences in employment prospects rather than differences in program 
quality. They also objected that a comparison between community college programs and private 
technical programs was unfair because the missions of the two types of institutions are different 
Community college programs, because they include general education as one of their goals, 
focus less on job training and placement than do technical schools. 

Proponents of the program were not dissuaded by these arguments and stood fumly behind 
their belief that some accountability data, even if not perfect, was better than none. They believed 
that students had a right to know about differences among institutions, regardless of why the 
differences existed. They responded to the charge that the missions and content of different 
programs were not comparable by asserting that statewide consistency in curriculum would not 
be a bad idea. The outcome information, they believed, could encourage schools with less 
successful programs to investigate and adopt some of the methods and curricula of successful 
programs. Schools in depressed treas, proponents argued, would improve if they either cut 
programs that did not serve local needs or increased their efforts to find job placements outside 
the local area. 

Data Accuracy 

Opponents of the program argued that using the unemployment insurance data base would 
produce inaccurate results and therefore provide misleading information to potential students, 
who might then avoid pcrfectiy good programs whose low placement rates were caused by 
factors other than low quality. This argument was based on the recognized limitations of the 
unemployment insurance data — particularly that self-employed graduates and completers who 
continued their educations, left the state, or entered the military would appear unemployed. 
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To deal with this problem, the state investigated matching vocational school completion 
records with military enlistments and the rosters of the state four-year colleges so that students 
enrolling in further education or joining the military would not be counted among the 
unemployed. Although the program did not use these additional data sources in the first year, the 
state planned to incorporate them later. To further improve accuracy, Arizona hoped to add 
searches of unemployment insurance data bases in neighboring states such as Colorado and 
California if those states developed similar programs. 

"Ctecmung** 

Most institutions, public and private alike, complained that their placement rates were 
relatively low because their schools served difficult populations. The emphasis on placement and 
wages, they argued, would encourage **crcaming** — the recruitment of students most likely to 
succeed and failure to serve the low-skilled and disadvantaged most in need of services. 

The Governor's Office acknowledged this possibility, but countered that because no 
funding depended on performance and no sanctions were being proposed for low placement 
rates, schools would not be pressured to **cream.'*^o In addition, every school claimed to serve 
the most difficult students. Consequently, the state reasoned, no school would be at a 
disadvantage because of its clientele. 

Usefulness of the Data 

Finally, colleges questioned the usefulness of the data. First, they argued, the 
unemployment insurance information replicated outcome data already available to the state and 
institutions from the national VEDS system (now discontlriued). In fact, it did not — the VEDS 
data were virtually useless because they were so incomplete. A major advantage of the 
unemployment insurance data was that it was much more complete. 

Second, they argued that the data would not help students because student mobility was 
limited by the community college fee structure — fees are lower for students attending the college 
in their region. However, the State argued that the fee systr,m allowed students to attend any 
college in the stale for the same low fee if the local program did not meet their needs. The state 



^%istitutions would, however, be subject to pressure from emoUment changes if stu nts responded to the low 
placement rates and avoided such programs. With enrollments down, institutions wouid be hurt financially. It 
would be possible to avoid this problem by calculating placement rates separately for certain groups such as 
physically or learning disabled. 
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was confident that this rule would pennit students to choose colleges based on the data presented 
in the outco^nes report 

Support for Information Disclosure 

Although the general reaction of the conununity colleges was negative, some colleges, 
especially private institutions, welcomed the state's effort to collect student outcome data. Most 
institutions were interested in the success of their graduates but had not been able to track them 
well through follow-up surveys, which arc veiy expensive and usually have disappointingly low 
response rates. 

The legislature was easily sold on the information disclosure project as a good way to 
generate performance measures and introduce accountability. It also addressed one of the 
legislature's greatest concerns in education — that students be informed of the earnings potential 
of an occupation before going into debt with student loans. The supporters prevailed primarily 
because the main opponents, the community colleges, were undergoing a restructuring of their 
state aid and did not want to jeopardize their budget, being considered by the legislature at the 
time. Representatives of the community colleges testified against the project, but, in the words of 
one observer, testified very carefully, guaranteeing cooperation with any program that would 
help students. 

Other than the community colleges, the initiative had no organized opposition. With the 
support of the governor and sponsorship of the majority whip, the legislation passed. 

Implementation 

The Department of Economic Security (DES), which maintains the unemployment 
insurance data base, in some ways would have been the natural choice for the task of collecting 
and analyzing the data. The design team realized early on, however, that imposing this new, 
major responsibility on the department would meet strong opposition because of the extra work 
involved. The state therefore contracted with Northern Arizona Umversity to collect and tabulate 
the data from the institutions. DES had only to run the tape containing social security numbers 
and program codes against the employment data, a minimal burden. 

Although the system was operational for only one y« ar, Arizona's experience illustrates 
many of the measurement and interpretation issues that otl.i:; * states adopting a similar program 
could expect to encounter. The most important technical issues were choosing appropriate 
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outcome measures and dealing with data limitations. Other problems included releasing timely 
data, protecting individual privacy, and securing school cooperation. These issues and problems 
were approached cooperatively by a project advisory committee composed of representatives 
from community colleges, private institutions, the Department of Education, and the Department 
of Economic Security. The committee was assisted by state staff and an economist 

Outcome Measures 

The published data included three outcome measures: completion rates, placement rates, 
and wages. How institutions were to define completers was not clear, because there were no 
statewide guidelines for what constituted a program in any occupation. Nor was the state 
involved in certifying program completion. The advisory conmuttee therefore allowed each 
institution to decide what constituted a program and successful completion. Some schools 
adopted narrow definitions of completion and had fewer completers; others chose to adopt mote 
liberal definitions and had more completers. Placement rates were calculated by dividing the 
number of completers found in the unemployment insurance data base by the number of 
completers. Quarterly average wages were computed for completers based on one, two, three, or 
four quarters worth of data, depending on how many quarters they were employed. 

Data Limitations 

While the unemployment insurance data base is a valuable source of information on 
employment outcomes that can be accessed at very low cost, it does have serious limitations, 
some of which were described earlier. The limitations discussed here are the ones regarded as the 
most serious in Arizona. 

A major drawback was that it was not possible to determine whether or not students entered 
employment in the fields for which they were trained, only whether or not they were employed. 
In Arizona, the unemployment insurance files contain only an industry code for businesses, not 
occupational codes for individuals. The state could determine that a completer was woricing for a 
trucking company, for example, but not whether the student was a truck driver, mechanic, or 
janitor for that company. Because of this data limitation, Arizona decided to count any job as a 
placement. An alternative — considered but rejected — would have been to try to identify which 
occupations went with each industry. 

A second major area of concern was that the unemployment insurance data offered definite 
outcome information only about vocational graduates who found employment in the state. There 
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are many activities that educators consider positive outcomes of vocational education that were 
not reflected in the Arizona data, including entering the military, becoming self-employed, 
continuing on to a four-year institution, or securing employment outside the state. Completers in 
these categories could not be distinguished from the unemployed. Consequently, placement rates 
may have appeared deceptively low in certain program areas. Fbr example, nuuiy cosmetologists 
are technically self-employed. They rent spaces from salons but are not employees of the salon 
owners. Graduates of certain techiucal programs may be mere likely to enter the military than are 
other students, introducing a downward bias in the job placement rates of those programs. 

Arizona's unemployment insurance Hies record only total wages earned in a quarter, not 
hourly wages. As a result, the average wages reported for an occupation mixed wages earned by 
both part-time and full-time workers. The average also combined the wages of those who 
woiked the full quarter and those who worked only part of th€ quarter. The wage data did not, 
therefore, accurately describe potential earnings in an occupation, especially when the reported 
averages were based on a small number of cases. 

Many programs had very small numben of completers each year, In order to protect the 
identity of individuals and to avoid reporting misleading results because of a small number of 
cases, Arizona decided not to publish information about programs with fewer than 25 
completers. Because most programs in the state turned out to have fewer than 25 graduates, the 
limit was later changed to 10. Even this standard resulted in the omission of many programs 
from the published report 

Privacy 

Protecting individual privacy was another major concern, as it would be for any program 
proposing to use social security numbers to locate people in confidential records. DES objected 
to releasing employment data in any form that allowed identification of individuals by outside 
agencies. To overcome this problem, DES conducted the computer match of social securit/ 
numbers with employment information and released only aggregate data. 

Timeliness 

The timeliness of the data was another source of concern. Arizona did not publish its 
outcome report until approximately two years after the end of the school year. Although nn.any 
would have liked more immediate feedback, the delay was unavoidable. The delay was not 
caused by unemployment insurance data, which are available shortly after the end of each 
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quarter, but by the requirement in the law that the report show placement and wages for the first 
full year after graduation. The law recognized that students could take at least a year to fmd a job. 

Cooperaaof 

The law required all public and private schools to submit, for all of their graduates, the 
social security numbers and the names of the occupations for which they were trained. It 
provided no sanctions for schools that failed to comply. The project's success depended on the 
schools' cooperation, but fean that institutions would refuse to provide data proved to be 
unfounded. All of the community colleges and most of the private technical schools provided the 
required information. 

Dissemination 

After collecting and tabulating the completion, placement, and wage data on vocational 
students, NAU produced a fm^d report. The whole project had cost NAU so much more than 
originally anticipated that funds available for disseminating the results were severely limited. In 
the end, the report was distributed to state agencies, vocational institutions, and high school 
counselors, but not to the public. 

Although dissemination was not targeted directly at potential students, they did have access 
to the findings through their high school counselors and NAU assumed that the outcome 
information would be publicized by schools anxious to advertise their own placement and 
average earnings data. However, the published report was not presented in the best format for 
students and parents. It was published as a rather official-looking document on standard report 
paper. If more funding had been available, the university would have printed a simpler version of 
the findings in brochures for distribution to high school students and their parents. 

Arizona has no information about how many students saw the report or the extent to which 
potential vocational students used the information to inform their choices. A few students 
contacted NAU directly to get the report, and presumably high school counselors made it 
available to other students. Since the Arizona program lasted only one year, neither NAU nor the 
governor's office could analyze the effect the information had on student choice. Project staff 
estimated that it could take three to four years for vocational training outcome data to become 
integrated into high school advising, student choice, and institutional planning. 
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Institutional ile8po'.ise 

Although NAU was not able to evaluate systematically the response of vocational 
instinitions to the report, it was able to collect some anecdotal evidence of changes stimulated by 
the information disclosure requirement A few examples can serve to illustrate some of the 
responses that occuned. 

The reporting requirements forced institutions to look carefully at how their programs were 
defined and what was required for completion. One college, for example, restructured a two-year 
occupational program into two programs — a one-year program and and two-year program — in 
response to low completion rates in the two-year program. The two-year program, which is often 
taught in one year at private schools, lost many students after one year because they had acquired 
enough skills to enter the labor market The program was reorganized so students could enroll in 
cither a one-year program or a two-year program. With this restructuring, the college could count 
those getting jobs after only one year as successful completers. Because funding was based on 
enrollment, the coCege had a financial incentive to keep students for two years. The publishing of 
the placement information motivated the college to meet the needs of students who wanted to 
enter the labor force sooner, saving these students a year's tuition and time. 

Anotiier reaction to the law was an increased focus on placement at the public institutions. 
Colleges had placement services, but many were insufficient The new focvxs on placement gave 
colleges the incentive to improve or expand placement services and many colleges responded by 
increasing resources allocated to placement activities and by working harder to place students. 

The new legislation also motivated some instructors to increase their contacts with industry. 
Knowing their prograiii:' performance would be evaluated on job placements, instructors moved 
to contact employers and update their curriculum in order to improve the job-relevance of their 
courses and identify potential job opportunities for their students. 

The data provided useful information that the state could have used to assess the funding of 
vocational programs. For example, the report revealed a number of programs with very high 
enrollments in some courses but small numbers of completers. Investigation showed that a 
number of these programs included courses that were counted as vocational for funding 
purposes, but that were in fact large lecture courses in vocational education departments, taken as 
elcctives by many non-vocational students. Since vocational courses receive 40 percent in 
additional funding per full-time equivalent student because equipment costs and smaller class 
sizes make vocational education more expensive, the state could see that schools were increasing 
their funding by defining low-cost lecture courses as vocational courses. 
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Inforniation Disclosure Costs 

The legislation appropriated $20,000 for NAU to collect and publish the outcome data. 
When they started, NAU researchers considered this ample funding for the project During the 
first year, however, unanticipated expenses raised the total cost to approximately $50,000 and 
korced the university to use its own general funds to complete the project The costs incurred by 
the Department of Economic Security — mainly for computer time — were minimal (only about 
$2,000 to $3,000), becc'ise the computer program to match completion information to the 
unemployment insurance data files could be tacked onto other programs and run at very little 
additional expense. 

Over time, the total cost of the information disclosure program in Arizona would have 
declined. First-year fixed costs for planning the project and developing computer programs were 
high and would not recur. However, additional funding would have been necessary to match the 
school completion files to postsecondary education files and military records. Also, to 
communicate the findings to parents and potential students effectively, it would have been 
necessaiy to write and distribute a brochure aimed at this audience. 

The Program's Demise 

After the fu^t vocational education report. Governor Babbitt left office and was replaced by 
Governor Mechami Reportedly, relations between the new governor and NAU were strained, at 
best The new Governor cut the state education budget significantly, including the NAU budget 
In the budget negotiations with the university, the governor rcjwrtcdly refused to directly fund 
the inforniation disclosure project In a move to force the university to use funds effectively, the 
governor told NAU to fund the project directly out of its own general funds if it was a priority. 
The vocational education unit at NAU, which had invested a significant amount of its own 
money in the project the previous year, decided it could not afford to continue the research 
without direct funds. Although the law remains on the books, the project fell dormant without 
NAU's participation. 

Adding to the project's political problems, information disclosure never had an organized 
constituency. The main beneficiaries of the program were potential vocational students and their 
parents. Many of these people were unaware of the legislation, and the group had no organized 
representative in state government 

Although the lack of funding directly caused the demise of Arizona prograrA, public and 
legislative awareness of technical difficulties were contributing factors. Opponents of the 
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program made sure that these difficulties were well covered in the press. The technical problems 
that caused the most concern were the large number of programs wiih fewer than ten completers 
for which no information was published and the fact that the program could not match 
employment occupations to the field of training. In the words of one observer, "the tech A 
problems did not kill the project, but they sure didn't help." 

CONCLUSIONS 

Our consideration of the potential benefits of information disclosure and our review of 
Arizona's experience suggest that improving information about the expected outcomes of 
educational alternatives is feasible and chat improved information has a good chance of improving 
student choice and the quality of postsecondary vocational education. More definitive 
conclusions about the impact of info -iaation disclosure must await evaluation of a program that 
has been in effect for several years. 

Because Arizona's information disclosure project was so short-lived, it was not possible to 
evaluate the impact on student choice. Arizona's experience did show, however, that when an 
information disclosure project begins, it creates a new forum for schools and educational 
agencies to consider the quality of postsecondary vocational educaUon. An information 
disclosure project gives states a reason tc address issues such as the consistency of occupational 
programs across the state. The publication of placement and wage data encourages schools to 
consider important issues, including the need for stop-out certificates, the responsiveness of 
programs to local employment opportunities, the relevance of curriculum to skills demanded by 
local industry, and the cffectivciiess of student placement services. 

Arizona's use of unemployment insurance data as a source of outcome information can 
serve as a model for other states. These data can provide accurate information on placement and 
wages at relatively low cost. States must be aware of the limitations inherent in these files, 
however. Of particular concern are the absence of occupation codes for individual's jobs, the 
lack of information on wage rates, and missing data on students not employed in jobs covered by 
the state's unemployment insurance program. If the limits are well understood in advance, states 
can avoid frustrated expectations. Florida has demonstrated that it is possible to compensate for 
at least some of the limitations by searching other data bases and by surveying employers to 
obtain information on occupations. States might also consider requiring employers to include 
occupations and wage rates for newly hired employees in their unemployment insurance reports. 
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The limitarions of unemployment insurance data suggest that institutions and state agencies 
should not rely entirely on this data for information on outcomes. It cannot replace in-depth 
infomiation about program quality that is gathered from site visits, seif-reviews, the accreditation 
process* and other sources of information available to states and schools. Unemployment 
insurance information cannot replace students surveys, because only students themselves caii 
report whether they use the skills they learned in school in their jobs, whether they are 
voluntarily unemployed or working part time, and whether they are satisfied with the training 
they received in vocational programs. What unemployment data offers that these other data do 
not is a low-cost means of gathering information on all former students, statewide, including 
those the schools cannot cont^t because no currert address is available and those who would not 
respond to z questionnaire. 

There are many ways to use unemployment insurance data in conjunction with other 
sources of data. With a little creativity — as has been exhibited in Florida — states can greatly 
enhance the quantity and quality of outcome information. Arizona explored (although did not 
punue) several strategies for tracking the outcomes of students by using other data bases such as 
military enlistments, four-year institution student data bases, and unemployment insurance data 
bases in other status. States could also use the unemployment insurance dita base to locate 
former students so tney can be contacted for surveys. 

Unemployment insurance data bases are also potentially useful for investigating long-term 
outcomes of vocational education. For example, the earnings of vocational completers could be 
tracked indefinitely — at least as long as they remain in the state. Research could also be 
conducted on job mobility and employment security based on the number of times that persons in 
each occupational area change jobs. Researchers could even determine whether students worked 
before entering programs and how their wages compared before and after training. For students 
not entering postseccmdary vocational programs immediately after high school, wages sometimes 
fall after graduation when they enter new fields. Unemployment insurance data would allows us 
to see how long it takes wages to rise to the before-school rate, and how long it takes trainees to 
recoup their tuition costs. 

Although the potential of these data is exciting, an important lesson to learn from Arizona's 
experience is to expect resistance to the publication of outcome-related information. The 
implementation process must be carefully thought out. In Arizona, the advisory committee 
contributed to the high level of coopcraticm NAU experienced with the schools. Involving public 
schools, private schools, labor, education, legislative committees, and executive staff is 
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important to ensuring a sound methodology, cooperation in the field, and accepUncc of the 
information when it is released 

Another technique for political survival — one not used in Arizona — is a sound plan for 
evaluating the effects of an information disclosure project To get continued support for an 
information disclosure project, states should plan to document the results. Evaluation should 
monitor the reaction of students, schools, and organizations. It should assess the effectiveness of 
dissemination, discovering how many high school students sec the outcome report and whether 
or not the students use the information. The evaluation should assess the reactions of schools to 
see if the schools view the information as a benefit or an imposition, if school administrators plan 
changes in order to improve their performance on the measures, if instructors are awaie of the 
reports, and if placement services change to address student employment needs. This information 
could help states secure continued support for information disclosure projects and provide solid 
evidence for other states considering information disclosure programs. 
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